Efficient Structure and Motion: Path Planning, Uncertainty and Sparsity

This thesis explores methods for solving the structure-and-motion problem in computer vision, the recovery of three-dimensional data from a series of two-dimensional image projections. The first paper investigates an alternative state space parametrization for use with the Kalman filter approach to simultaneous localization and mapping, and shows it has superior convergence properties compared with the state-of-the-art. The second paper presents a continuous optimization method for mobile robot path planning, designed to minimize the uncertainty of the geometry reconstructed from images taken by the robot. Similar concepts are applied in the third paper to the problem of sequential 3D reconstruction from unordered image sequences, resulting in increased robustness, accuracy and a reduced need for costly bundle adjustment operations. In the final paper, a method for efficient solution of bundle adjustment problems based on a junction tree decomposition is presented, exploiting the sparseness patterns in typical structure-and-motion input data

View Planning and Refractive Modeling for Structure and Motion

This thesis presents contributions to structure-and-motion estimation, a central topic in the field of geometric computer vision. In particular, the problem of view planning is considered, and continuous and discrete optimization-based algorithms are given for how to plan the path of a sensor to its destination, while balancing the competing goals of path length and reconstruction accuracy. The same concepts are then applied to the problem of sequential 3D reconstruction from unordered image sequences. By propagating reconstruction uncertainties and actively selecting the order in which images are used via view planning, significant gains in robustness and computational efficiency are achieved. The second topic of the thesis is refractive structure-and-motion, specifically the problem of absolute pose estimation when the camera and structure are separated by an optically refracting plane. Using methods from algebraic geometry for solving multivariate polynomial systems, efficient minimal and near-minimal solvers are constructed. Finally, a practical method for calibrating a set of cameras under refraction is given, including an algorithm for efficient refractive bundle adjustment

Covariance Propagation and Next Best View Planning for 3D Reconstruction

This paper examines the potential benefits of applying next best view planning to sequential 3D reconstruction from unordered image sequences. A standard sequential structure-and-motion pipeline is extended with active selection of the order in which cameras are resectioned. To this end, approximate covariance propagation is implemented throughout the system, providing running estimates of the uncertainties of the reconstruction, while also enhancing robustness and accuracy. Experiments show that the use of expensive global bundle adjustment can be reduced throughout the process, while the additional cost of propagation is essentially linear in the problem size

Absolute pose for cameras under flat refractive interfaces

This paper studies the problem of determining the absolute pose of a perspective camera observing a scene through a known refractive plane, the flat boundary between transparent media with different refractive indices. Efficient minimal solvers are developed for the 2D, known orientation and known rotation axis cases, and near-minimal solvers for the general calibrated and unknown focal length cases. We show that ambiguities in the equations of Snell's law give rise to a large number of false solutions, increasing the complexity of the problem. Evaluation of the solvers on both synthetic and real data show excellent numerical performance, and the necessity of explicitly modelling refraction to obtain accurate pose estimates

Discrete Optimal View Path Planning

This paper presents a discrete model of a sensor path planning problem, with a long-term planning horizon. The goal is to minimize the covariance of the reconstructed structures while meeting constraints on the length of the traversed path of the sensor. The sensor is restricted to move on a graph representing a discrete set of configurations, and additional constraints can be incorporated by altering the graph connectivity. This combinatorial problem is formulated as an integer semi-definite program, the relaxation of which provides both a lower bound on the objective cost and input to a proposed genetic algorithm for solving the original problem. An evaluation on synthetic data indicates good performance

A step towards self-calibration in SLAM: Weakly calibrated on-line structure and motion estimation

We propose a structure and motion estimation scheme based on a dynamic systems approach, where states and parameters in a perspective system are estimated. An online method for structure and motion estimation in densely sampled image sequences is presented. The proposed method is based on an extended Kalman filter and a novel parametrization. We derive a dynamic system describing the motion of the camera and the image formation. By a change of coordinates, we represent this system by normalized image coordinates and the inverse depths. Then we apply an extended Kalman filter for estimation of both structure and motion. Furthermore, we assume only weakly calibrated cameras, i.e. cameras with unknown and possibly varying focal length, unknown and constant principal point and known aspect ratio and skew. The performance of the proposed method is demonstrated in both simulated and real experiments. We also compare our method to the one proposed by Civera et al. and show that we get superior results

Optimal View Path Planning for Visual SLAM

In experimental design and 3D reconstruction it is desirable to minimize the number of observations required to reach a prescribed estimation accuracy. Many approaches in the literature attempt to find the next best view from which to measure, and iterate this procedure. This paper discusses a continuous optimization method for finding a whole set of future imaging locations which minimize the reconstruction error of observed geometry along with the distance traveled by the camera between these locations. A computationally efficient iterative algorithm targeted toward application within real-time SLAM systems is presented and tested on simulated data

Combining Foreground / Background Feature Points and Anisotropic Mean Shift For Enhanced Visual Object Tracking

This paper proposes a novel visual object tracking scheme, exploiting both local point feature correspondences and global object appearance using the anisotropic mean shift tracker. Using a RANSAC cost function incorporating the mean shift motion estimate, motion smoothness and complexity terms, an optimal feature point set for motion estimation is found even when a high proportion of outliers is presented. The tracker dynamically maintains sets of both foreground and background features, the latter providing information on object occlusions. The mean shift motion estimate is further used to guide the inclusion of new point features in the object model. Our experiments on videos containing long term partial occlusions, object intersections and cluttered or close color distributed background have shown more stable and robust tracking performance in comparison to three existing methods